Territorial bias in university rankings: a complex network approach

University rankings are increasingly adopted for academic comparison and success quantification, even to establish performance-based criteria for funding assignment. However, rankings are not neutral tools, and their use frequently overlooks disparities in the starting conditions of institutions. In this research, we detect and measure structural biases that affect in inhomogeneous ways the ranking outcomes of universities from diversified territorial and educational contexts. Moreover, we develop a fairer rating system based on a fully data-driven debiasing strategy that returns an equity-oriented redefinition of the achieved scores. The key idea consists in partitioning universities in similarity groups, determined from multifaceted data using complex network analysis, and referring the performance of each institution to an expectation based on its peers. Significant evidence of territorial biases emerges for official rankings concerning both the OECD and Italian university systems, hence debiasing provides relevant insights suggesting the design of fairer strategies for performance-based funding allocations.


Assortativity scatter plots of OECD university networks
The scatter plots reported in Supplementary Fig. S1 provide a qualitative representation of the assortativity of the territorial network (left panels) and the educational offer network (right panels), with respect to the overall score and the specific dimensions of the THE rating. In each scatter plot, the point coordinates correspond to the THE scores of pairs of connected institutions, while the point color is determined by the strength of connections. Remarkably, the scatter plots concerning the territorial network, especially those related to the overall score, citations and international outlook dimensions, show a typical assortative pattern: most of the high-weight dots lie close to the main diagonal line, that represents connections between universities with the same THE score. On the other hand, in the scatter plots associated with the educational offer network, dots indicating strong connections are much more evenly distributed, for all the THE ratings. These results indicate that the territorial network shows a stronger tendency to exhibit connections between nodes with similar THE scores, compared to the educational offer one. . Assortativity scatter plots of OECD university networks with respect to the overall score and the specific dimensions of the THE rating. In these scatter plots each dot corresponds to one of the links in the territorial (left panels) and educational offer (right panels) OECD university network, and its coordinates along the horizontal and vertical axes represent the THE scores, for a specific ranking, of the two universities at either end of that link. A configuration in which high-weight links are mostly distributed close to the main diagonal, thus connecting universities with similar THE scores, reveals the existence of an assortative pattern, absent in the case of a more uniform distribution of weights.

Community detection workflow in OECD university networks
In this work, we perform community detection by a hierarchical algorithm, that starts finding a firstlevel partition of the network, and then proceeds to finding the partition of first-level communities, treated as sub-networks. The criteria to choose the optimal partition and the iteration level of the hierarchical process are discussed in the Materials and Methods section.
The first hierarchical level of subnational area community detection yields the partition resulting from the subdivision of OA1 in (OA1a,OA1b) with 96% agreement and 〈 〉 = 0.957 for = 0.9, of OA2 in (OA2a,OA2b) with 95% agreement and 〈 〉 = 0.995 for = 0.95, and of OA3 in (OA3a,OA3b) with 100% agreement for = 0.9. At the third level, the algorithm finds a partition in which some communities have a smaller than 5% share of the network, therefore the iteration stops at the second level. A geographical representation of the accepted partition is provided in Figure S2. To construct a partition in communities of the OECD universities, we associate each university with the region of its main seat, and collect all institutions related to geographical areas belonging to the same OA community. Therefore, we obtain the final subdivision, reported in the Results section of the main text: Concerning the educational offer network, at the first level the network splits into three communities at = 1, with a rather reduced agreement (59%), but still with large mutual similarity between them (〈 〉 = 0.957); the resulting partition reads: • OE1: 301 universities with an educational offer focused on engineering, economics and science; • OE2: 427 universities with a very generalized range of educational offer; • OE3: 360 universities with an educational offer characterized by underrepresented engineering areas; The second-level partition is still acceptable: for = 1, OE1 splits in two communities (OE1a,OE1b) with 100% agreement, OE2 in two communities (OE2a,OE2b) with 96% agreement and 〈 〉 = 0.973, and OA3 in three communities (OE3a,OE3b,OE3c) with 93% agreement and 〈 〉 = 0.947. The final partition, reported in the Results section of the main text, reads: The iteration stops at the second level also in this case, since at the third level the algorithm finds a partition in which some communities have a smaller than 5% share of the network. The full list of universities with their community membership in both the territorial and educational offer networks is reported in Supplementary Data S1.

Debiasing parameters and principal component analysis for THE ranking dimensions
In Supplementary Fig. S3, we show the scatter plots in the planes ( ! , " ) of the debiasing parameters referred to each THE ranking dimension. Points are colored according to their territorial community membership. In all scatter plots, each dot corresponds to an OECD university, and its coordinates along the horizontal and vertical axes represent the debiasing parameters. The values ! and " , referred to the specific dimension, assess the results achieved by the institution in the corresponding THE ranking, respectively by comparison with the rest of the OT and OE community it belongs to. Each dot in the scatter plot is colored according to its OT community membership.
We report in Table S1 the details of Principal Component Analysis (PCA) on the distribution of debiasing parameters associated to THE dimensions, namely the variance explained by each component and the Pearson correlation of each component with the GDP per capita PPP of the OECD subregion in which the main university seat is located. The related p-values are obtained by comparing the empirical Pearson correlation with the exact distribution of the correlation values between two random vectors, independent and normally distributed.  .

Assortativity scatter plots of Italian university networks
The scatter plots shown in Supplementary Fig. S4 provide a qualitative representation of the assortativity of the territorial network (left panels) and the educational offer network (right panels), with respect to the overall score and the specific dimensions of the CENSIS rating. The distribution of points and colors in the scatter plots allows to investigate possible effects of the context (both geographical and educational) on the performance of universities in CENSIS rankings. Results are analogous to the ones already discussed for the OECD university environment: in the Italian case study, the territorial network tends to be more assortative than the educational offer one. In particular, pronounced assortativity patterns are observed in the territorial network with respect to the overall score and some specific dimensions, such as employability and international outlook, among the others. Assortativity scatter plots of Italian university networks with respect to the overall score and the specific dimensions of the CENSIS rating. In these scatter plots each dot corresponds to one of the links in the territorial (left panels) and educational offer (right panels) Italian university network, and its coordinates along the horizontal and vertical axes represent the CENSIS scores, for a specific ranking, of the two universities at either end of that link. A configuration in which high-weight links are mostly distributed close to the main diagonal, thus connecting universities with similar CENSIS scores, reveals the existence of an assortative pattern, absent in the case of a more uniform distribution of weights.

Community detection workflow in Italian university networks
Hierarchical community detection is performed for the Italian university networks, in the same spirit and with the same procedure as in the OECD case. The first hierarchical level of community detection on the subnational area network provides the following partition: • IA1: 30 provinces in the center-north of the country, including Rome; • IA2: 23 provinces in the center-south of the country, with unanimous agreement (therefore, 〈 〉 = 1), for all resolutions in [0.8,1] Such a partition reflects once more the historical and economic gap between North and South of Italy. At the second level, we find another partition with a dominant interpretation in terms of size and geographical location: • IA1a: 21 provinces in the center-north with a small administrative center; • IA1b: 9 provinces in the center-north, mostly with a large or historically relevant administrative center; • IA2a: 10 provinces in center-south and Sardinia; • IA2b: 13 provinces in the south.
The second-level partitions of IA1 in (IA1a,IA1b) and of IA2 in (IA2a,IA2b) are reached with 100% agreement at = 0.95 and = 1, respectively. The iteration stops at this stage since the partitions with largest consensus of second-level communities, returned at the next level, are trivial. A geographical representation of the accepted partition is provided in Figure S5. As in the OECD case, we associate each university with the province of its main seat, and collect all institutions related to provinces belonging to the same subnational area community. Therefore, we obtain the final subdivision, reported in the Results section of the main text: • IT1a: 25 universities in center-north provinces with a small administrative center; • IT1b: 35 universities in center-north provinces, mostly with a large or historically relevant administrative center; • IT2a: 13 universities in center-south and Sardinia; • IT2b: 19 universities in the south.
As concerns the educational offer network, the approach here adopted for community detection returns the following subdivision, reported in the Results section of the main text: • IE1: 31 small, telematic universities, oriented to law, economics or foreign languages; • IE2: 44 medium-to-large general-purpose universities; • IE3: 9 polytechnic and small engineering-oriented universities; • IE4: 8 research hospitals and health-oriented small universities, with unanimous agreement (hence, 〈 〉 = 1) at = 1. The partition at the following level is rejected, as it contains communities with less than 5% of nodes of the whole network. The full list of universities with their community membership in both the territorial and educational offer networks is reported in Supplementary Data S3.

Debiasing parameters and principal component analysis for CENSIS ranking dimensions
In Supplementary Fig. S6, we show the scatter plots in the planes ( ! , " ) of the debiasing parameters referred to each CENSIS ranking dimension. Points are colored according to their territorial community membership.
(e) CENSIS international outlook (f) CENSIS employability In all scatter plots, each dot corresponds to an Italian university, and its coordinates along the horizontal and vertical axes represent the debiasing parameters. The values ! and " , referred to the specific dimension, assess the results achieved by the institution in the corresponding CENSIS ranking, respectively by comparison with the rest of the IT and IE community it belongs to. Each dot in the scatter plot is colored according to its IT community membership.
We report in Table S2 the details of PCA on the distribution of debiasing parameters associated to CENSIS dimensions, namely the variance explained by each component and the Pearson correlation of each component with the average per capita available income of the province in which the main university seat is located. The related p-values are obtained by comparing the empirical Pearson correlation with the exact distribution of the correlation values between two random vectors, independent and normally distributed.  . ±

Application of Gaussian Mixture model to the principal components of the debiasing parameters associated to the CENSIS overall score
To investigate the possible multimodal nature of the principal components of the debiasing parameters associated to the CENSIS overall score, we use a family of unidimensional Gaussian Mixture models, characterized by a number # of components that varies from 1 to 10. We report here for completeness the detailed outcomes of the Shapiro-Wilk test (87) and of the AIC and BIC score minimization (88), which is applied if the Shapiro-Wilk test is passed for more than one # : • For PC1, all the components properly pass the Shapiro-Wilk test only in the case # = 1. For # = 6,7,8,9, some components pass the test, while it cannot be performed on other components, since they have less than 4 elements.
• For PC2, all the components properly pass the test for all the # 's from 1 to 5, and from 7 to 10, while the case # = 6 involves components with less than 4 elements. The models that minimize the AIC and BIC scores are those with # = 1 and # = 2, respectively. Specifically, # = 1 minimizes the BIC (−137.26, against −131.06 of # = 2), while # = 2 minimizes the AIC (−142.58, against −141.87 of # = 1). In any case, the result for # = 2 allows to estimate the bias, evaluated as the horizontal distance 0.147 between the peaks of the two Gaussians.

Data collection for the OECD academic system
In this section, we provide information on data used to construct the OECD university networks. In Table S3, we introduce a list of the territorial indicators, compiled by the Organisation for Economic Co-operation and Development (OECD), used for the construction of the international territorial area network. Table S4 reports a list of the categories used by Times Higher Education to compile their rankings by educational area: this classification provides the basis to construct the educational offer network.  Mortality rates for the 0 to 4 years old population (deaths per 10 000 people) Municipal waste rate (kilos per capita) Net firm creation rate (%) (firm birth rate minus firm death rate) Number of motor road vehicles per 100 people Over-qualification rates for the foreign-born (%) Part-time employment incidence (%) Patent applications (PCT) per 1 000 000 people Percentage of early leavers from education and training, for the 18 to 24 years old population Percentage of foreign-born among the total population Percentage of household expenses dedicated to housing costs Percentage of households with broadband internet access Percentage of labour force with at least secondary education Percentage of labour force with at least tertiary education Percentage of people exposed to more than 10 µg/m³ (micrograms per cubic metre) of PM2.5 Percentage of people satisfied with the availability or quality of healthcare Percentage of population from 15 to 19 years old enrolled in public or private institutions Percentage of population from 25 to 64 years old participating in education and training Percentage of population from 25 to 64 years old with at least tertiary education Percentage of population satisfied with affordability of housing Percentage of population satisfied with efforts to deal with poverty Percentage of population satisfied with efforts to preserve the environment Percentage of population satisfied with quality of air Percentage of population satisfied with quality of water Percentage of population satisfied with roads and highways Percentage of population satisfied with the quality of public transportation systems Percentage of population that believe corruption is spread throughout the government in the country Percentage of population that believe women are treated with respect and dignity in their country Percentage of population that believes their place of residence is a good place to live for gay or lesbian people Percentage of population that believes their place of residence is a good place to live for migrants Percentage of population that believes their place of residence is a good place to live for racial and ethnic minorities Percentage of population that feel safe walking alone at night around the area they live Percentage of population that have been assaulted or mugged in the previous 12 months Percentage of population that have confidence in judicial system and courts Percentage of population that have confidence in the local police force Percentage of population that have confidence in the national government Percentage of population with a disposable income below the 60% of national median disposable income Percentage of population with a disposable income below the 60% of regional median disposable income Percentage of total electricity production that comes from coal Percentage of total electricity production that comes from fossil fuels (natural gas and oil, excluding coal) Percentage of total electricity production that comes from nuclear power Percentage of total electricity production that comes from renewable sources Percentage of young population (from 18 to 24 years old) not in education, employment or training (

Data collection focusing on the Italian academic system
In this section, we provide information on data used to construct the Italian university networks. In Table S5, we introduce a list of the territorial indicators, compiled by the Italian National Institute of Statistics (ISTAT), on which the territorial area network is based. Each indicator is reported along with the reference year from which its value is retrieved. In Table S6, we report the classification of the Italian degree categories (classi di laurea) in terms of "broad fields" of the International Standard Classification of Education (ICSED). Indicators that do not pass the redundancy selection process are highlighted in light red; the indicator "Average per capita income (EUR)", that is removed to constitute a benchmark for the wealth of provinces, is highlighted in light blue. 2018 Tickets for museums belonging to circuits (% of total) Table S6. Classification according to the International Standard Classification of Education (ICSED) of the Italian degree categories. Information is used for the construction of the Italian educational offer network. Both bachelor (L) and master (LM) degree courses are included (80,81).

Default parameters of the Spin Glass community detection algorithm
Community detection is performed using the Spin Glass algorithm (85,86) of the igraph Python library (igraph). While the variational parameters are discussed in the main text, we report here the choices made for the default parameters: • spins is the parameter that sets the upper limit for the number of communities. It is fixed to the number of nodes in the network or in the previous-level community that is going to be partitioned, according to the stage of the hierarchical algorithm. An exception is represented by the OECD educational offer network, since the algorithm does not support a number of communities as large as 1088; therefore, only in that case the value of spins is set to 1088/4 = 272 communities.
• implementation is a value that can be set to either default, if one wants to use the faster original implementation, or to neg, if one wants to take into account negative weights; the parameter is set to neg (the choice is irrelevant in the case of the educational offer networks). • lambda_ is the argument that specifies the balance between the importance of present and missing negatively-weighted edges within a community. Smaller values of lambda lead to communities with less negative intra-connectivity. This value is set to 0.01, and the results are very stable with respect to its variations. • update_rule specifies the null model of the simulation. Possible values are config, which sets the model as a random graph with the same vertex degrees as the input graph, and simple, in which the model is a random graph with the same number of edges. The value is set to config. • cool_fact represents the cooling factor for the simulated annealing process that leads to the optimal configuration. The value is set to 0.5, with the results being very weakly dependent on it.

List of supplementary files
Data S1. (separate file) Full list of OECD universities with their community membership in both the territorial and educational offer networks.

Data S2. (separate file)
Full list of OECD universities with the debiasing parameters ( ! , " ) and the related principal component values PC1 and PC2, referred to the THE overall score and to all its dimensions.

Data S3. (separate file)
Full list of Italian universities with their community membership in both the territorial and educational offer networks.

Data S4. (separate file)
Full list of Italian universities with the debiasing parameters ( ! , " ) and the related principal component values PC1 and PC2, referred to the CENSIS overall score and to all its dimensions.